REFINEMENT INEQUALITIES AMONG SYMMETRIC DIVERGENCE 

MEASURES 



INDER JEET TANEJA 

Abstract. There are three classical divergence measures in the literature on infor- 
mation theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson- 
Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic - geometric mean di- 
vergence. These bear an interesting relationship among each other and are based on log- 
arithmic expressions. The divergence measures like Hellinger discrimination, symmetric 
X 2 — divergence, and triangular discrimination are not based on logarithmic expressions. 
These six divergence measures are symmetric with respect to probability distributions. 
In this paper some interesting inequalities among these symmetric divergence measures 
are studied. Refinement over these inequalities is also given. Some inequalities due to 
Dragomir et al. |BJ are also improved. 



1. Introduction 



Let 



(pi,P2,-,Pn) 



Pi > 0,^pi = 1 > , n ^ 2, 



be the set of all complete finite discrete probability distributions. For all P,Q <G T n , the 
following measures are well known in the literature on information theory and statistics: 

• Hellinger Discrimination 

n 

(1.1) HP\\Q) = 1 - B(P\\Q) = - £ (VPi- V^) 2 , 

i=l 

where 

(1.2) B(P\\Q) = y/Mi, 
is the well-known Bhattacharyya |T] coefficient. 

• Triangular Discrimination 

(1.3) A(P||Q) = 2 [1 - W{P\\Q)\ = T (P '~ 9i)2 , 

Pi + Qi 

where 



;i.4) W{P\\Q) = Y J 



Zpiqi 



i=1 P* + 
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is the well-known harmonic mean divergence. 
• Symmetric Chi-square Divergence 

(1-5) *(P||Q) = x 2 {P\\Q) + x 2 (Q\\P) = j2 iPl ~ qi)2{Pl + qt) 



i=i 



PiQi 



where 
(1.6) 



x\p\\Q) = Y1 



i=i 



(ft - (lif = spp[ _ 1 



is the well-known x s ! — divergence (Pearson |!Uj " 
• J-Divergence 



;i.7) 



J(P||g) = 5^(p i - ft )ln( 



ft 



i=l 



Jensen-Shannon Divergence 



I(P\\Q) = 2 



Pi In 



2ft 



ft + Qi 



i=l 



2% 



ft + <?i 



Arithmetic-Geometric Mean Divergence 



(1.9) 



T(P\\Q) = J2 



i=l 



Pi + q% 



In 



ft + gi 
2 V / Pi* 



After simplification, we can write 

;i.io) j(p||q) = 4[/(p||q) + t(p||q)]. 

The measures I(P\\Q), J(P\\Q) and T(P||Q) can be written as 
J(P\\Q) = K(P\\Q) + K(Q\\P), 



(1.11) 
(1.12) 
and 
(1.13) 
where 
(1.14) 



I(P\\Q) 



T(P\\Q) 



K P 



,P + Q 



+ K[Q\\ 



P + Q 



K 



P + Q, 



P +K 



P + Q 



\\Q 



K(P\\Q) = ^ Pi log 



i=i 



is the well known Kullback-Leibler |J relative information. 

We call the measures given in (jl.lj) . ()1.3jl . (|1.5p . (|1.7jl . ()1.9|) and (jl.lOJ) as symmetric 
divergence measures, since they are symmetric with respect to the probability distributions 
P and Q. The measure is due to Hellinger [7|. The measure (jl.5|) is due to Dragomir 
et al. jE|, and recently has been studied by Taneja [T5] . The measure (jl.7j) is due to 
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Jeffreys and later Kullback-Leibler studied it extensively. Some times it is called 
as Jeffreys-Kullback-Leibler's J- divergence. The measure (jl.8|) is due to Sibson jTTj, and 
later Burbea and Rao [21 0] studied it extensively. Initially, it was called as information 
radius, but now a days it is famous as Jensen-Shannon divegence. The measure (J1.9j) 
is due to Taneja jT3], and is known by arithmetic- geometric mean divergence. For one 
parametric generalizations of the measures given above refer to Taneja [T2|[^]. A general 
study of information and divergence measures and their generalizations can be seen in 
Taneja [HEUl EE]. 

In this paper our aim is to obtain an inequality its improvement in terms of above 
symmetric divergence measures. This we shall do by the application of some properties 
of Csiszar's f— divergence. 

2. Csiszar's /-Divergence 

Given a function / : [0, oo) — > R, the f-divergence measure introduced by Csiszar's |4] 
is given by 

(2-1) ^(P||Q) = f>/(-\ 

i=i ^ qi/ 

for all P, Q G r n . 

The following theorem is well known in the literature. 

Theorem 2.1. (Csiszar's jHEJj. If the function f is convex and normalized, i.e., f(l) = 
0, then the f— divergence, Cf(P\\Q) is nonnegative and convex in the pair of probability 
distribution (P, Q) G T n x T n . 

Recently, Taneja established the following property of the measure (j2.1|) . 

Theorem 2.2. (Taneja jT^jj. Let f\, f% : / C IR+ — > R two generating mappings are 
normalized, i.e., /i(l) = /2(1) = and satisfy the assumptions: 

(i) fi and f 2 are twice differentiable on (a,b); 

(ii) there exists the real constants m, M such that m < M and 

(2.2) m ^ ^ M, f»(x) > 0, Vx G (a, b), 
then we have the inequalities: 

(2.3) m C h (P\\Q) ^ C h {P\\Q) ^ M C h (P\\Q). 
Proof. Let us consider the functions rj m . s (-) and r] Ms {-) given by 

(2.4) r] m (x) = fi(x) - m f 2 (x), 
and 

(2.5) r ]M {x) = M f 2 {x)- h{x), 

respectively, where m and M are as given by (|2.2j) . 

Since f\{x) and f 2 (x) are normalized, i.e., /i(l) = /2(f) = 0, then T] m (-) and t]m{') are 
also normalized, i.e., rj m (l) = and r] M {l) = 0. Also, the functions fi(x) and f 2 (x) are 
twice differentiable. Then in view of (J2.2j) . we have 

(2-6) v'L(x) = nix) - m f' 2 \x) = f 2 \x) (%® -m^0, 
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and 

(2.7) rf M (x) = M f'(x) - f»(x) = f»{x) (m - |M 
for all x G (r, P). 

In view of (|2.6|) and (|2.7|) . we can say that the functions rj m (-) and t?m( - ) given by (|2.4|) 
and (|2.5|) respectively, are convex on (r,R). 
According to Theorem 12.11 we have 

(2.8) C Vm (P\\Q) = C h „ mh {P\\Q) = C h {P\\Q) - m C h {P\\Q) > 0, 
and 

(2.9) C VM (P\\Q) = C Mh - h (P\\Q) = M C h (P\\Q) - C fl (P\\Q) > 0. 
Combining and we have the proof of □ 

Now, based on Theorem 12. 11 we shall give below the convexity and nonnegativity of the 
symmetric divergence measures given in Section 1. 

Example 2.1. (Hellinger discrimination) . Let us consider 

(2.10) A(x) = ^(v^-1) 2 , xe(0,oo), 

in ( OOP , t/ien we nave C/(P||Q) = /i(P||Q), where h(P\\Q) is as given by M.l\) . 
Moreover, 

fh{x) = ^/T' 

and 

< 2 - n > = 

JTius we nave //^(s) > /or all x > 0, and hence, fh{x) is strictly convex for all x > 0. 
Also, we have /^(l) = 0. In view of this we can say that the Hellinger discrimination given 
by M.l)) is nonnegative and convex in the pair of probability distributions (P, Q) G T n x T n . 

Example 2.2. (Triangular discrimination) . Let us consider 

(2.12) f A (x)= (X ~ 1)2 ,xG(0,oo), 

x + 1 

in JjH)) ; tfien we nave C/(P||Q) = A(P||Q), wnere A(P||Q) is as given by ( TO)) . 
Moreover, 

_ (x- l)(x + 3) 
/a(X) " (x + l)2 ' 

and 

(2.13) /£(z) 



T/ras we nave f^{x) > /or a// x > 0, and hence, f/\{x) is strictly convex for all x > 0. 
j4/so 7 we have /a(1) = 0. In view of this we can say that the triangular discrimination 
given by J773J) is nonnegative and convex in the pair of probability distributions (P, Q) G 



r?i x T n . 
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Example 2.3. (Symmetric chi-square divergence). Let us consider 

(2.14) U(x) = {X - 1)2 X {X + 1} , x G (0, oo), 

in / TO) , then we have C f (P\\Q) = ^(P\\Q), where #(P\\Q) is as given by / TOP . 
Moreover, 

, , , _ (x - l)(2x 2 + x + 1) 

and 

(2.15) fUx) = 2ir ' +[) 



3 



X 

Thus we have fy(x) > for all x > 0, and hence, f^(x) is strictly convex for all 
x > 0. Also, we have /W(l) = 0. In view of this we can say that the symmetric chi-square 
divergence given by hi. 1^1 is nonnegative and convex in the pair of probability distributions 
(P,Q)eT n xT n . 

Example 2.4. (J-divergence). Let us consider 

(2.16) fj(x) = (x- l)lnx, x e (0,oo), 

in \2.1\) . then we have Cf(P\\Q) = J(P\\Q), where J(P\\Q) is as given by \l.l\j . 
Moreover, 

f'j(x) = 1 — x" 1 + lnx, 

and 

(2.17) /»(*) ' 1 



2 



X 

Thus we have f"(x) > for all x > 0, and hence, fj(x) is strictly convex for all x > 0. 
Also, we have fj(l) = 0. In view of this we can say that the J-divergence given by \1. 7| ) 
is nonnegative and convex in the pair of probability distributions (P, Q) G T n x T n . 

Example 2.5. (JS- divergence). Let us consider 

(2.18) fj(x) = | In i + In (^yj , x 6 (0, oo), 

in \2. 1\) . then we have Cf(P\\Q) = I(P\\Q), where I(P\\Q) is as given by lll.fy) . 
Moreover, 



2 \x+ 1 

and 

(2.19) f'j'(x) 



2x(x + 1)' 



Thus we have f"{x) > for all x > 0, and hence, fi(x) is strictly convex for all x > 0. 
Also, we have //(l) = 0. In view of this we can say that the JS-divergence given by hi. 8) 
is nonnegative and convex in the pair of probability distributions (P, Q) G T n x T n . 
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Example 2.6. (AG-Divergence) . Let us consider 
(2.20) f T ( x) = (?+l) ]n (pl\ . , e(0 ..x 



2v^. 

m ( QE J)) , £/ien we have Cf(P\\Q) = T(P\\Q), where T(P\\Q) is as given by M.ty) . 
Moreover, 

( x + l X 



1 - x~ l + 2 In 



and 

(2-21) fUx) = t + 1 

Thus we have f^x) > for all x > 0, and hence, fr(x) is strictly convex for all x > 0. 
Also, we have /t(1) = 0. In view of this we can say that the AG-divergence given by M.y\) 
is nonnegative and convex in the pair of probability distributions (P, Q) G T n x T n . 

3. Inequalities Among the Measures 

In this section we shall apply the Theorem 12 .2l to obtain inequalities among the measures 
given in Section 1. We have considered only the symmetric measures given in (jl.lj) . (|1.3|) . 
(US)), (fT1)-(IO|). 

Theorem 3.1. The following inequalities among the divergence measures hold: 

(3.i) \hp\\Q) < i{p\\Q) < KP\\Q) < < r(^IIQ) < 

The proof of the above theorem is based on the following propositions, where we have 
proved each part separately. 

Proposition 3.1. The following inequality hold: 
(3-2) \a(P\\Q)^I(P\\Q). 
Proof. Let us consider 

,x , ^ n (*o (^+i) 2 c rn v 

(3.3) ^ = 0)~> x 6(0,oo), 

where //(x) an d /a( x ) are as given by (j2.19j) and (|2.13|) respectively. 
From (|3.3j) . we have 

In view of ([3.4)1 . we conclude that the function gitJyx) is decreasing in x G (0, 1) and 
increasing in x G (1, oo), and hence 

(3.5) m = sup #/a(x) = #/a(1) = ~- 

xe(o,oo) 4 

Applying the inequalities (|2.3|) for the measures A(P||Q) and I(P\\Q) along with (|3.5|) 
we get the required result. □ 
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Proposition 3.2. The following inequality hold: 

(3.6) I(P\\Q)^h(P\\Q). 
Proof. Let us consider 

(3.7) gjh{x) = iM = ^ * e (°'°°)< 

where f"(x) and f^{x) are as given by (j2.19j) and ()2.11|) respectively. 
From (|3.7|) . we have 

, x — 1 I ^ 0, x ^ 1 

In view of ()3.8|) . we conclude that the function gih(x) is increasing in i 6 (0, 1) and 
decreasing in x G (1, oo), and hence 

(3.9) M= sup Ih (aO = g Ih (l) = 1. 

xg(0,oo) 

Applying the inequalities (|2.3|) for the measures I(P\\Q) and /i(P||Q) along with ()3.9|) 
we get the required result. □ 

Proposition 3.3. The following inequality hold: 

(3.10) h{P\\Q) < \j{P\\Q). 
Proof. Let us consider 

/ \ f/O) 4(x +1) , 

(3.11) 9Jh{x) = 'W) = ^/x U ' XE ^°°^ 

where f'j(x) and f'^x) are as given by (|2.17|) and (j2.11|) respectively. 
From ()3.11|) we have 

(3.12) ^ w =h^)/>». 

In view of (I3.12|) . we conclude that the function gjh{x) is decreasing in x G (0, 1) and 
increasing in x G (1, oo), and hence 

(3.13) m = inf g Jh (x) = g Jh {^) = 8. 

a;G(0,oo) 

Applying the inequalities (|2.3J) for the measures h(P\\Q) and J(P\\Q) along with ()3.13|) 
we get the required result. □ 

Proposition 3.4. The following inequality hold: 

(3.14) ±J(P\\Q)^T(P\\Q). 
Proof. Let us consider 

/ x / x f'K x ) 4(x+l) 2 

(3.15) gj T (x) = = x2 + 1 , x G (0, oo), 

where f"(x) and fx(x) are as given by (j2.17j) and (|2.21jl respectively. 
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From (J3.15|) we have 

In view of (13. 16)1 we conclude that the function gjx{x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

(3.17) M= sup g JT {x) = g JT {l) = 8. 

xe(o,oo) 

Applying the inequality (|2.3|) for the measures J(P\\Q) and T(P||Q) along with ()3.17|) 
we get the required result. □ 

Proposition 3.5. The following inequality hold: 

(3-18) T{P\\Q) < ±9{P\\Q). 

Proof. Let us consider 

(3.19) = = 8(^)^1) - * 6 (»•<»)■ 

where /^(s) and /^(rc) are as given by (|2.21|) and (|2.15|) respectively. 
From (|3.19|) we have 

, , , ( ) (x-l)(x 4 + 4x 2 + l) (^0, x<l 

In view of (|3.2U|) we conclude that the function grm^x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

(3.21) M= sup g T y{x) = 5r *(l) = — . 

ze(o,oo) J-t) 

Applying the inequality (|2.3|) for the measures T(P\\Q) and ^(P||Q) along with (|3.21|) 
we get the required result. □ 

The proof of the inequalities given in (|3.1|) follows by combining the results given in 
(Q, CT - (pTinjl . 5.14) and (l3~T%|) respectively. 

Dragomir et al. [H] proved the following two inequalities involving the measures (|1.3|) . 
(fT3|) and 

(3-22) < ±J(P||Q) - A(P\\Q) < ^*(P||Q), 

and 

(3.23) < i*(P||Q) - J(P||Q) < \d*{p\\q), 



where 



(p< 



\ i 



(3.24) jD *(P||Q) = ^ , _ 

In the following section we shall improve the inequalities given in (|3.1|) . An improvement 
over the inequalities 1)3.22)1 and ()3.23j) along with their unification is also presented. 
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4. Difference of Divergence Measures 
Let us consider the following nonnegative differences: 



(4.1) 


Dvt(P\\Q) 


= —V(P\\Q)-T(P\\Q), 


(4.2) 


D*j(P\\Q) 




(4.3) 


D m (P\\Q) 


= ^(p||g)-MP||Q), 


(4.4) 


D m (P\\Q) 


= l*(p||Q)-j(p||g), 


(4.5) 


D 9A (P\\Q) 


= ^*(P||Q)-iA(P||Q), 


(4.6) 

(4.7) 
(4.8) 


Dtj(P\\Q) 

D Th (P\\Q) 
Dti{P\\Q) 


= r(P||Q)-ij(P||g), 

= T(P||Q)-MP||Q), 
= T(P||g)-/(P||Q), 


(4.9) 


Dta(P\\Q) 


1 

= T(p||g)- i A(p||g), 


(4.10) 


D., h {P\\Q) 


= ij(p||g)-fc(p||g), 


( A 1 1 "\ 

(4.11J 


P > ji{P\\Q) 


^ 7/' Dl 1 /T\ fY Dl 1 /H\ 

= gJ{P\\Q) ~ i{P\\Q)> 


(A 19"! 

V / 

(4.13) 


n , a ( pi \n\ 


- up\\n\ a(pwd') 

= /»(p||Q)-j(p||g) l 


(4.14) 


^A(p||g) 


= fc(p||g)-^A(p||g) J 


and 






(4.15) 


^/a(p||Q) 


= /(p||g)-iA(p||g). 



In the examples below we shall show the convexity of the above measures fl4.1|) - ()4.15|) . 
In view of Theorem 2.1 and Examples 2.1-2.6, it is sufficient to show the nonnegativity 
of the second order derivative of generating function in each case. 

Example 4.1. We can write 

d*t(p\\Q) = ^(P\\Q) - t(p\\Q) = X>iW (-), 



where 



hr(x) = —U (x) - fr (x) , x > 0. 
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Moreover, we have 

(4.16) nr(x) = ^(x)-f^(x) 

x 3 + l x 2 + l (x- l) 2 (x 2 + X + 1) 

= = — > n Vr > n 

8x 3 4x 2 (x + l) 8x 3 (x+l) " ' 

where (x) and (x) are as given by \2.1ty and \2.21\) respectively. 
Example 4.2. We can write 

1 1 - 

D 9J (P\\Q) = ^(P\\Q) - ^J(PWQ) = 5>/« ' 

u>nere 



i6 8 Vft 



= (») - g/j (x) , x > 0. 



Moreover, we have 



(4.i7) /;;W=i/;w-i/;w 



+ 1 I + n (I _ !).(, + i) te > 



x 3 x 2 ) 8x 3 

where (x) and /" (x) are as (jroen 6y \2.1ty and \2.11 ) respectively. 
Example 4.3. VKe can write 



D* h {P\\Q) = ^>{P\\Q) - h(P\\Q) = 



Pi 

16 v "~ v/ v "~ v/ z — ' \ o. 



wnere 

/*ft(») = 77:/* (») - A (x) , x > 0. 
16 

Moreover, we have 
(4.18) /M = ^(x)-A(x) 

1 /x 3 + 1 1 \ {Xy/X- if 



, -. - , ., ^ 0, Vx > 0, 

4 \ 2x 3 Xy/x J 8x 3 

where fl (x) ana 1 /£' (x) are as given by \2.1ty and 112.11)) respectively. 
Example 4.4. We can write 

d*i(p\\Q) = ^9{P\\Q) - I(P\\Q) = (f)' 

u>nere 

= ~/a> (&) - fi{x), x> 0. 
16 
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Moreover, we have 
(4-19) fh(x) = (x) - f'j' (x) 



1 fx 3 + l 1 ^ _ (x- l) 2 (x 2 + 3x + l) 



2x \ Ax 2 x + lj 8x 3 (x + 1) 

where (x) and f" (x) are as given by and \2.19i) respectively. 

Example 4.5. We can write 



1 1 n 

D*a(P\\Q) = — *(^||Q) - -A(P||Q) = J>/* A 



Pi 
u>aere 

/*A(ac) = | Q/* (a;) - /a (ar)^ , x > 0. 

Moreover, we have 

(4.20) /£ aW = I g/J (x) - /I (,)) = ^1 - ^ 

(x - l) 2 (x 4 + 5a; 3 + 12x 2 + 5x + 1) 

= teJ(ITTp >o,Vx>o, 

where (x) and / A (x) are as ow en by i2.15}) and \2.1§) respectively. 
Example 4.6. We can write 

D TJ (P\\Q) = T(P\\Q) - \j{P\\Q) = J2 qi f T j fa), 

u>aere 

/tj(jc) = /t (a:) - -/j (x) , x > 0. 

o 

Moreover, we have 

(4-21) = g# (*) 

x 2 + l x + 1 (x-1) 2 

^ 0, Vx > 0, 



4x 2 (x + 1) 8x 2 8x 2 (x + 1) 
where f!f (x) and f'j (x) are as given by \2.21)) and \2.11\) respectively. 
Example 4.7. We can write 



D Th (P\\Q) = T(P\\Q) - h(P\\Q) = 

i=i 

where 

hh{x) = h (a;) - f h (x), x> 0. 
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Moreover, we have 



(4.22) AW ^ W _ /aiH l(_^_^) 



( v /x~-l) 2 (x + v /x~+l) 



^ 0, Vx > 0, 



4x 2 (x + 1) 

where f T (x) and f% (x) are as given by \2.21)) and \2.11\) respectively. 
Example 4.8. We can write 

D TI (P\\Q) = T{P\\Q) - I(P\\Q) = f^qJ TI (|) , 

where 

fxi(x) = fx (x) - fi (x) , x > 0. 

Moreover, we have 
(4.23) % I (x) = ft{x)-fl(x) 

x 2 + l 1 (x-1) 2 



^ 0, Vx > 0, 



4x 2 (x + l) 2x(x + l) 4x 2 (x + l) 
where f' T (x) and f" (x) are as given by \2.21)) and \2.19i) respectively. 
Example 4.9. We can write 

Dta(P\\Q) = T(P\\Q) - U(P\\Q) = X>/ TA (-), 

4 i=i 

where 

Ixa(x) = f T (x) - -f A (x) , x > 0. 

Moreover, we have 

(4.24) M = /;w-k(^ 1 1 



4^ v ' 4x 2 (x + 1) (x + l) 3 
= (x-l) 2 (x 2 + 4x + l) 

4x 2 (x + l) 3 ^ ' ' 
where f T (x) and (x) are as given by \2.21\) and \2.1$) respectively. 
Example 4.10. We can write 



n 

D Jh {P\\Q) = -J(P\\Q) - h(P\\Q) = J2<lif. 



n 

Pi 

Jh 



i=l 

where 



Moreover, we have 



fjh(x) = ~fj (x) - f h (x) , x > 0. 



(4-25) tf h (s) = -/£(*) 



£+i-^- = ^- 1 ) a > o,y g >o > 

8x^ 4xv^ 8x 2 
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where f'j (x) and fft (x) are as given by \2.11\f and \2.11\) respectively. 
Example 4.11. We can write 



Dn{P\\Q) = \j(P\\Q) - I{P\\Q) = iTqifn 

where 

fji( x ) = g/j 0) ~fi(x),x> 0. 

Moreover, we have 
(4-26) = (*)-//(*) 

x + 1 1 _ (^- 1) 2 

~ ~8~^~ ~ 2x{x + l) ~ 8x 2 {x + 1) ^ ' > ' 
where f" (x) and f" (x) are as given by \2.11\f and \2.19i) respectively. 
Example 4.12. We can write 

Dja(P\\Q) = \j{P\\Q) - ~A(P||Q) = |>/ JA (j) , 

where 

Jja(x) = - fj (x) - -/ A (x),x> 0. 

Moreover, we have 

(4.27) /^nkw-kw ' 1 2 



4' ,AV y 8x 2 (x + 1) 3 
x- l) 2 (x 2 + 6x+ 1) 



> 0, Vx > 0, 



8x 2 (x + l) 3 

where f" (x) and f'£ (x) are as given by \2.11\f and \2.13j) respectively. 
Example 4.13. We can write 



D hI {P\\Q) = h(P\\Q) - I(P\\Q) = J^Qifh 
where 

fhi(x) = f h (x) - // (x) , x > 0. 

Moreover, we have 
(4.28) Ki(x) = fh(x)-f'i'(x) 

4^ 2s(x + 1) 4x 3 / 2 (x + 1) ^ ' 
where f£ (x) and f" (x) are as given by \2.11)) and \2.19i) respectively. 
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Example 4.14. We can write 



1 n 
D hA (P\\Q) = h(P\\Q) - -A(P\\Q) =Y,<hfh* 



i=i 

where 



fhA(x) = f h (x) - ~f A (x),x> 0. 



Moreover, we have 



(4.29) M = /;w-k(^ 



4 A v ' Axy/x (x + 1) 



{^-lf (y/x + lY (X + 1) + 4X 



> 0, Vx > 0, 



4x 3 / 2 (x + l) 3 

where f£ (x) and (x) are as aiuen 6t/ \2.11)) and \2.1ty) respectively. 
Example 4.15. We can write 

1 n / 

D IA (P\\Q) = I(P\\Q) - -A(P\\Q) = - 



//a(^) = // (a?) - | /a (a;) , x > 0. 



Moreover, we have 



(4.30) m = /;w-^w 



2x(x + l) (x + 1) 3 2x(x + l) 3 
where f" (x) and f'£ (x) are as given by \2.19\i and \2.1ty) respectively. 

Thus in view of Theorem 12.11 and Examples I4.1M.15( we can say that the divergence 
measures given in (|4.1j) - (|4.15J) are all nonnegative and convex in the pair of probability 
distributions (P, Q) G T„ x T n . 

5. Refinement Inequalities 
In view of (13. the following inequalities are obvious: 

(5.1) D* T (P\\Q) < D*j(P\\Q) ^ D m (P\\Q) ^ D^(P\\Q) ^ D* A (P\\Q), 

(5.2) D TJ (P\\Q) <: D Th (P\\Q) ^ D TI (P\\Q) <: D TA (P\\Q), 

(5.3) D Jh (P\\Q) < DniPWQ) ^ D JA (P\\Q) 

and 

(5.4) D hI (P\\Q)^D hA (P\\Q). 

In view of the relation ([1.10)1 . we have the following equality: 

(5.5) DniPWQ) = l -D TI (P\\Q) = D TJ (P\\Q). 
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In this section our aim is to establish refinement inequalities improving the one given 
in (J3.1|) . This refinement is given in the following theorem. 

Theorem 5.1. The following inequalities hold: 

(5.6) D IA (P\\Q) ^ 2 -D hA {P\\Q) <: 2D hI (P\\Q) ^ D TJ (P\\Q), 

(5.7) D IA (P\\Q) <: ^D hA (P\\Q) <: ~D JA (P\\Q) <: l -D TA (P\\Q) <: D TJ (P\\Q), 
and 

(5.8) D TJ (P\\Q) <: 2 -D Th (P\\Q) <: 2D Jh (P\\Q) < ^D 9A (P\\Q) 

3 

^ l -D m {P\\Q) <: ~D m (P\\Q) <: 1d 9J (p\\Q) ^ ±DMP\\Q), 

The proofs of the inequalities ()5.6|) - (j5.8J) are based on the following propositions. 
Proposition 5.1. We have 

(5.9) D IA (P\\Q)^~D hA (P\\Q). 



Proof. Let us consider 

/ \ fJLM 2 v /i(x-l) 2 / 

9lA.hA(x) = — — - = X £ 1 

JhA\ X ) [X + 1) - 8 (y/x) 

2y/x~{^x~ + I)' 



(y/x + if (X + 1) + 4 X 




for all x G (0, oo), where f'/ A {x) and fh A (x) are as given by ()4.3()|) and ()4.29j) respectively. 

Calculating the first order derivative of the function gi A _h A (x) with respect to x, one 
gets 

(y/x + 1) (x 5 / 2 - 2x 3 / 2 + 3x 2 + 2x - 3^ - 1) 

(5.10) 9j\h\\ x ) = 9 L 

3 v ^[x 2 + 6x + 2 v ^(a; + l) + l] 2 

(x-l)(x + l) (x + 4y^+l) 

y/x [x 2 + 6x + 2yfx~ (x + 1) + l] 2 

In view of (I5.1()j) we conclude that the function gi A _h A (x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

2 

(5.11) M = sup gi A _hA(x) = gmjiAW = ^■ 

xe(o,oo) 3 
By the application of Q with (I5~TTT) we get (Q). □ 

Proposition 5.2. We have 

(5.12) ^a(P||Q) ^3D w (P||Q). 
Proof. Let us consider 

/ x /a'aOe) (x + 1) (v^+ l) 2 + 4x 
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where fH&(x) and fhj(x) are as given by (|4.29|) and (|4.28|) respectively. 

Calculating the first order derivative of the function ghA.hi(x) with respect to x, one 
gets 

, 4a: 3/2 + x 2 _ 4^ _ 1 

1°- 1<3 J 9hAJiI\ x ) — ~, ~T73 



x (x + iy 

x-l)(x + 4y/x+l) J > 0, X<1 



^/x(x + iy [<0, x>\ 

In view of f!5.13|) we conclude that the function ghAJii{x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

(5.14) M = sup g h A.hi(x) = ^a_w(1) = 3. 

xe(o,oo) 

By the application of Q with (I5~T1) we get (ETHl . □ 

Remark 5.1. In view of Propositions [E~l\ and XETib. and the inequality \3. 1)) we conclude 
that 

(5.15) I{P\\Q) ^ \h{P\\Q) + ^A(P||Q) < h(P\\Q). 
Proposition 5.3. We have 

(5.16) D hI (P\\Q) ^ \d tj {P\\Q). 



9hi.tj{x) = 4^ = - ^— — 2 , x G (0, oo) 



Proof. Let us consider 

where f'^x) and fj>j(x) are as given by ()4.28j) and ()4.21|1 respectively. 

Calculating the first order derivative of the function ghi_Tj{x) with respect to x, one 
gets 

, . y/x- 1 J > 0, x < 1 

(5-17) 9hi.Tj( x ) 



y/x(^/x + 1) [< 0, x > 1 

In view of ()5.17|) . we conclude that the function ghi.Tj(x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

(5.18) M= sup g h i.Tj{x) = g h i.Tj{l) = \- 

xe(o,oo) 2 

By the application of (Q with (15~T%D we get (IBToT) . □ 

Remark 5.2. In mew o/ Propositions f3T31 and £/ie inequality \3. 1)) we conclude the fol- 
lowing inequality 

(5.19) /»(P||Q) < ±J(P\\Q) + \l{P\\Q) ^ \j{P\\Q). 
Combining the inequalities ()5.9j) . ()5.12j) and (|5.1fij) we get ()5.6j) . 
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Proposition 5.4. We have 



(5.20) A, A (P||Q) ^A, A (P||Q). 



Proof. Let us consider 



fl M - - 2y^ [(x + l) 3 ~ 8s 3 / 2 ] 

2v/x [(</x + l) 2 (x + 1) + 4x 



(v^+ l) 2 (x 2 + 6x + 1) 

for all x G (0, oo), where /^ A (x) and f"&{x) are as given by (|4.29J1 and ()4.27|) respectively. 

Calculating the first order derivative of the function g/i A _j A (x) with respect to x, one 
gets 

(5.21) g' A m (x) = ^ r [3x 4 - 4x 3 - 18x 2 - 12a; - 1 

feA - J v^(v^+l) 3 (x 2 + 6x + l) 2 L 

(x 4 + 12x 3 + 18x 2 + 4x - 3)] 

(^/x- 1) (x+ l) 2 (x 2 + Ax^/x + 14x + 4v/i+ 1) J> 0, x < 1 
v^(v^+ l) 3 (x 2 + 6x + l) 2 \<0, x>l' 

In view of (|5.21jl we conclude that the function <?/i A _j A (x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

3 

(5.22) M = sup g h A.jA(x) = ^a.ja(1) = -■ 

xe(0,oo) 4 

By the application of flZ3) with flEU) we get (jEH- □ 

Remark 5.3. In f«ew of 'Proposition ^ 4\ and the inequality \3.1\) we conclude the following 
inequality 

(5.23) h(P\\Q) ^ ^J(P\\Q) + ^A(P||Q) ^ l -J{P\\Q). 
Proposition 5.5. We have 

(5.24) D JA (P\\Q)^^D TA (P\\Q). 
Proof. Let us consider 

/ \ fjA(x) x 2 + Qx+l 

9ja - ta(x) = I^aJx) = 2(x 2 + 4x + iy x (0 ' 00)1 

where /j A (x) and /r A (x) are as given by (|4.27|l and (|4.24J) respectively. 

Calculating the first order derivative of the function gjAjrA^x) with respect to x, one 
gets 

, f , (x-l)(x + l) />0, x<l 
(5 - 25) ^ (x) = ' (x 2 + 4x + l) 2 <0, x>T 
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In view of (|5.25|) we conclude that the function gjASTA^x) is increasing in x G (0, 1) 
and decreasing in x G (1, oo), and hence 

2 

(5.26) M = sup gjA.TA(x) = #jajta(1) = -• 

xG(0,oo) <J 

By the application of Q with lET2fll) we get (l5"21) . □ 
Proposition 5.6. VKe /iai>e 

(5.27) £>ta(P||Q) <3D TJ (P||Q). 
Proof. Let us consider 

/ \ /ta(^) 2(x 2 + 4x + 1) , n , 

' TA - Tj(x) = €M = \x + iy 1 G (0 ' oo) ' 

where /^a( x ) an d frj( x ) are as gi ven by (|4.24|) and (j4.21|) respectively. 

Calculating the first order derivative of the function grA.Tj(x) with respect to a;, one 
gets 

(5.28) gWl) = _i(£L_il(>°. *<|. 

(x + iy i i < o, x > i 

In view of (I5.28|) we conclude that the function gTA.Tj(x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

(5.29) M = sup g T A.Tj(x) = g T A_T.j(X) = 3. 

By the application of (Q with (E| we get (ET27I) . □ 

Remark 5.4. In mew o/ Propositions E^l and \Kl\ and the inequality \H. 1\) we conclude 
the following inequality 

(5-30) ^(P||Q) < |r(P||Q) + ^A(P||Q) < T(P||Q). 

Combining the inequalities (j5T9JI . (|5~24l and (|5~27f) . we get (|5~7j) . 

Proposition 5.7. VKe /jaue 

(5.31) D TJ {P\\Q)^D Th {P\\Q). 
Proof. Let us consider 

/ \ _ ItA*) _ (x - l) 2 

~ " 2^+i- 2v ^(x+i)]' x ^ 1 

(v^+1) 2 



2{x + 1)' 

for all x G (0, oo), where f?j(x) and fxh( x ) are as given by ()4.21j) and ()4.22|) respectively. 

Calculating the first order derivative of the function grj.Th{.x) with respect to x, one 
gets 

(y/x-l)(y/x + l) f>0, x< 1 



(5.32) gTj_Th(x) 



Ay/X (X + y/x + 1) I < 0, £>1 
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In view of (I5.32j) we conclude that the function gTj_Th(%) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 

2 

(5.33) M = sup g T j.Th(x) = g T j_Th(X) = o- 

ieG(0,oo) 3 

By the application of (J27J) with (1q~33]) we get (l5~3Tf . □ 
Proposition 5.8. VKe /iave 

(5.34) D T?l (P||Q)<3L> Jft (P||g). 
Proof. Let us consider 

_ _ 2[x 2 + l- v /i(x + l)] 

/jfcW (x + 1) (V^ - 1; 

_ 2(x+y/x + l) 
X + l 

for all x G (0, oo), where /r/i( x ) an d fjh( x ) are as gi ven by ()4.22j) and ()4.25|) respectively. 

Calculating the first order derivative of the function gTh_jh(%) with respect to x, one 
gets 

, x- 1 /< 0, x < 1 

(5.35) *"» W = - ■/?(« + !)' \<0, «>r 

In view of ()5.35|) we conclude that the function gTh.Thi.%) is increasing in a; G (0, 1) and 
decreasing in x G (1, oo), and hence 

(5.36) M = sup g T h.jh(x) = g T h.Jh{l) = 3. 

z£(0,oo) 

By the application of (Q with (15361) we get (JQ1|) . □ 

Remark 5.5. In view of Propositions 377 and \5.tA and the inequality h3. 1)) we conclude 
the following inequality 

(5 .37) m\Q)* nm) \ m) *\mm. 

Proposition 5.9. We have 

(5.38) D Jh (P\\Q)^±D* A (P\\Q). 
Proof. Let us consider 

, ^ fJM _ x(y^-l) 2 (x + l) 3 



x(x + 1) 



3 



(y/x + if (x 4 + 5x 3 + 12x 2 + 5x + 1) 



for all x G (0, oo), where f'j h {x) and f1 A {x) are as given by (|4.25jl and ()4.2()j) respectively. 
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Calculating the first order derivative of the function gjhja/±{x) with respect to x, one 
gets 

\2 




(5.39) 9jhJSA( x ) — ~r~p : ; 3 / ~; 77-T7 7 772 x 

(y/x + 1) (x 4 + 5x 3 + 12x 2 + 5x + 1) 

x [x 5 + 5x 4 + 6cc 2 (\/S - l) 2 + 5x + 1 
+yfx (x 4 + 3x 3 + 4x 2 + 3x + l)] . 

From (|5.39jl . one gets 

(5-40) sfjhjaeS? 

In view of f)5.4Dj) we conclude that the function gjhjn^{x) is increasing in x G (0, 1) and 
decreasing in x G (1, 00), and hence 

(5.41) M = sup g Jh jBA(x) = 9jh.9a0) = j^- 

xe(o,oo) 1^ 

By the application of (Q with (l5~4Tj) we get ijOfy . □ 

Remark 5.6. In i>zeu> 0/ Propositions and the inequality hS. 1\) we conclude the fol- 
lowing inequality 

(5.42) \j(P\\Q) + \&(P\\Q) < ±9t{P\\Q) + 12h(P\\Q). 
Proposition 5.10. We have 

(5.43) D 9A {P\\Q)^^D m {P\\Q). 

5 

Proof. Let us consider 



f& A (x) x 4 + 5x 3 + 2x 2 + 5x + 1 



g * A - mix) = im = (x+mxi+sx+i) ' x G (0 ' 00)5 

where f1 A (x) and f^j(x) are as given by (|4.2()jl and ()4.19|) respectively. 

Calculating the first order derivative of the function g^AjSfi( x ) with respect to x, one 
gets 

, 4x(x-l)(2x + l)(x + 2) />Q, x<l 

(5.44) W)- (x + l)3(a;2 + 3x + l)2 \< 0, x>l 

In view of (I5.44j) we conclude that the function g^Ajai(x) is increasing in x G (0,1) and 
decreasing in x G (1, 00), and hence 

6 

(5.45) M= sup g^AjSfi(x) = £?*a_*i(1) = 7- 

xe(o,oo) 

By the application of fl2H| with (fOKj) we get (jQ3|) . □ 

Remark 5.7. In view of Propositions \5.1(\ and the inequality \H. 1\) we conclude the 
following inequality 



(5.46) I{P\\Q)^\ 



^(p\\q) + ^a(p\\q) 



<^(P\\Q)- 



REFINEMENT INEQUALITIES 



21 



Proposition 5.11. We have 

(5.47) D m {P\\Q) < ^D 9h (P\\Q). 

Proof. Let us consider 

, . f^(x) (x- l) 2 (x 2 + 3x + l, 
Hh\ x ) {x + 1) (iv 1 - 1) 

= (y/j+l) 2 (x 2 + 3x + l) 
(x + 1) (x + y/x + l) 2 

for all x G (0, oo), where f^j(x) and f'^ h (x) are as given by (J4.19)) and (|4.18jl respectively. 

Calculating the first order derivative of the function g^i_^h{x) with respect to x, one 
gets 

(x - 1) (3x + y/j + 3) [>0, x<l 
(x + s/x+1) (x+1) 2 (>0, x>l 

In view of (15.48)1 we conclude that the function gmjuhix) is increasing in x G (0, 1) and 
decreasing in x G (1, 00), and hence 

(5.49) M = sup gmjsih{x) = gmji>h{l) = 77- 

xe(o,oo) y 

By the application of (Q with fl5H| we get ([CTTjl . □ 

Remark 5.8. In meu> of Propositions \5.11\ and the inequality hS. 1\) we conclude the 
following inequality 



1*(P||Q) + 9I(P||Q) 



^Y^{P\\Q)- 



(5.50) h (p\\Q)^L 
Proposition 5.12. We have 

(5.51) D m {P\\Q) ^-D*j{P\\Q). 

o 

Proof. Let us consider 

9 ^ AX) = lUx) = (v^+l) 2 (-+l)' * G ( ° ,00) ' 
where f^ h (x) and f^j(x) are as given by (|4.18|) and ()4.17|) respectively. 

Calculating the first order derivative of the function gmhja.j{x) with respect to x, one 
gets 

_ (y/i-lHx + y/^+l) [>0, X<1 

[b.b2 } g m _*Ax)- (v ^ +1)3(x + 1)2 | <0; x>1 

In view of ()5.52j) we conclude that the function g^hJ5/j{x) is monotonically increasing 
in x G (0, 1) and decreasing in x G (1, 00), and hence 

9 

(5.53) M = sup gm-*j(x) = ^.*j(1) = 

xe(o,oo) o 

By the application of (Q with (l533j) we get (15311 . □ 
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Remark 5.9. In view of Propositions \5.1 6 A and the inequalities \3. 1)) we conclude the 
following inequality 



(5.54) 

o 

Proposition 5.13. We have 
(5.55) 
Proof. Let us consider 



-y(p\\Q) + 8h(P\\Q) 



^^(P\\Q)- 



D*j(P\\Q) < -D* T (P\\Q). 



f'ij{ x ) 



(x + 1) 



x G (0, oo) 



f£ T (x) x 2 + x + l 

where f1j(x) and f1 T (x) are as given by (|4.17|) and (|4.16|) respectively. 

Calculating the first order derivative of the function g^j^rix) with respect to x, one 
gets 



(5.56) 



9qj_t>T\ X ) 



(x - l)(x + 1) j > 0, x < 1 



{x 2 + x + l) 2 <0, x>\ 



In view of f!5.56|) we conclude that the function g^jjar^x) is increasing in x G (0, 1) and 
decreasing in x G (1, oo), and hence 



(5.57) 



M 



sup gyj_y T (x) = ^j.*t(1) = i:- 



xG(0,oo) 

By the application of ()2.3|) with (j5.57|) we get ()5.55|) . 



□ 



Remark 5.10. In view of Propositions [5.131 and the inequality \3. 1)) we conclude the 
following inequality 



(5.58) 



T(P\\Q)^ 
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*(P\\Q) + 3J(P\\Q) 



^-*(P\\Q). 



Combining (fPTJl . (Q4l . 1ET3SI) . (lo~m (jQ7j) . (15311) and (1535]) we get flEU). Thus 
the combination of the Propositions 15. 1115.131 completes the proof of the Theorem 15.11 



6. Final Comments 

(i) In view of inequalities (fHTTHj) . (pTtTIJ) . (ET2311 . (|5~3%J) and (jEHED , we have the 

following improvement over the inequality ()3.1|) : 

(6.i) \hp\\Q) < /(PIIQ) < f^HG) + ^A(p||g) < fc(p||g) 

< ±J(P\\Q) + |j(^||Q) < ^T(PHQ) + h(P\\Q) 



32 



*(P||Q) + 3J(P||g) 
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(ii) For simplicity, if we write, the divergence measures given in (J4.1)) - (J4.15j) by Di — Di 5 
respectively, then the Theorem 15 . II resumes in the following inequalities: 



(a) D 15 ^ |D 14 < 2D 13 *C D 6 ; 



(b) D 15 ^ lD u < \D 12 < |D 9 ^ £> fl ; 

(c) D 6 <: §£) 7 *C 2£> 10 < ±D 5 < ±L> 4 ^ ^ \D 2 ^ iDj. 



(iii) Following the similar lines of the propositions given in section 5, we can easily 
prove the following inequality, 

(6.2) D W (P\\Q)^±D*(P\\Q). 

where D*(P\\Q) is as given by 

The inequality (|6.2j) together with Theorem 15 .11 gives us the following improve- 
ment over the inequalities ()3.22j) and (|3.23J) : 

(6.3) D JA (P\\Q) < ^DvjiPWQ) < §Atr(P||Q) ^ 



or equivalent ly, 



1 2 1 

2 3 96 



From the inequality ()6.3|) and item (ii) (b)-(c), we observe that there are many 
divergence measures in between Dja(P\\Q) and Dyj(P\\Q). Thus the inequality 
()6.3|) improves the results due to Dragomir et al. jHj. 



(iv) The inequalities (|5.42|) and (J5.54j) can be written as 



(6.4) 



±*(P\\Q) + 12h{P\\Q)-±A(P\\Q) 



1 

< - 
9 



±*(P\\Q)+8h(P\\Q) 



The middle inequalities of ()6.4)1 follow in view of (J5.7)) and (|5.8j) . 
(v) The inequalities (|5.5()jl and (|5.54jl can be written as 



(6.5) 



h(P\\Q)< 
1 



9 



1 

10 
1 

16 
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9(P\\Q)+9I(P\\Q) 



*(P\\Q) + 8h(P\\Q) 
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